Why structural equation modeling?

As researchers in HCI or in the social sciences, we study human properties (operationalized as variables) and how they are related.

• Human variables: Often impossible to measure directly.
• How: Ideally, what causes changes in a certain variable?

Measurement

From properties to variables…

Causal relationships?

If we research relationships between variables, we are often interested in causality. Structural equation modeling uses regression to model relationships between variables (or constructs). But regression does not imply causality.

The gold standard for examining causality in science is the randomized controlled trial. But in HCI research, we often only have observational data.

Causal inference and observational data?

Benefits of SEM

Structural equation modeling then allows us to examine two relationships at the same time:

Process of SEM estimation

We know take you through the process of SEM estimation as you would go through it in your own research.

1. Define your research question
2. Define structural model
3. Define measurement model
4. Gather data
5. Prepare data
6. Estimate model
7. Bootstrap model

Define RQ

• You start your process by thinking about your research questions.
• Imagine you want to know what causes adoption of your software?

UTAUT2

Unified Theory of Acceptance and Use of Technology [3]

UTAUT2 and Trello

We are interested in examing UTAUT2 for Trello, specifically our RQ is:

• What determines BI for using Trello?

UTAUT2 has a strong theory and evidence base. This gives us some support for causal interpretation.

Build a structural model

From your research questions, you will derive some hypotheses.

We hypothesize that Performance Expectancey (PE), Effort Expectancy (EE), and Social Influence (SI) influence BI.

Build a structural model

So far this is equivalent to a regular multiple regression.

But we might also hypothesize that

• the number of navigation errors influences EE
• and that number of successful tasks influence PE

These hypotheses can be directly translated into code. And this code can be visualized as a DAG.

New model

Build a measurement model

Now, we think about how we want to measure these constructs.

Luckily, for UTAUT2, there are some standardized items so that part is made easy for us.

Example - Behavioral Intention (Scale 1-7):

• INT1: I intend to use Trello in the next month.
• INT2: I predict I would use Trello in the next month.
• INT3: I plan to use Trello in the next month.

Data Dictionairy

Excursus: Measurement theory

There is a distinction between reflective and formative measurement.

• Reflective means the scale is the result of a latent construct (openness).

• Formative means the scale is the construct (e.g. IQ).

These measurement concepts can correspond to different mathematical estimation techniques.

Excursus: Measurement theory

Most PLS-based SEM use formative scales.

Different software frameworks may use different terminology (e.g. SmartPLS, SEMinR)

Build a measurement model

For this exercise, we assume that all our constructs are type A composite constructs.

Measurement models

Items (manifest variables) are often called as the abbreviation and a number (e.g. PE1).

Constructs (latent variables) can have a single or multiple items.

The names should correspond to the column names in our data.

The specified model

Gather data

This model can be used for preregistration (e.g. using OSF).

Data collection is simplified using the prepared scales (survey, experiments).

Here, we collected data for you [1].

Prepare data

• recode data (numerical data)
• treat missing data (or SEMinR will do it for you!)
• rename variables (naming scheme)

Estimate model

Two main questions:

• Do our items measure sensible constructs?
• Does our data support the hypothesized relationships?

Single function call in SEMinR

Estimation using SEMinR

Estimation Technique

SEMinR uses Partial-Least Squares (PLS) estimation

• No need for normally distributed data
• Smaller sample sizes produce usable results

Outputs

Measurement model:

• Loadings as \(\lambda\) or weights as \(\omega\) values

Structural model:

• Path coefficients as \(\beta\) values
• Variance explained as \(r^2\)

Are these values “significant”?

Bootstrap a model

What is bootstrappingg?

• Repeat the estimation process \(n\) times
• Determine empirically 95% confidence intervals for parameters
• Also provides p-values, using t-distribution (assumes normality)

Bootstrap a model